Deep learning for biomarker and outcome prediction in cancer

Machine learning in the form of deep learning (DL) has recently transformed how computer vision tasks are solved in numerous domains, including image-based medical diagnostics. DL-based methods have the potential to enable more precise quantitative characterisation of cancer tissue specimens routinely analysed in clinical pathology laboratories for diagnostic purposes. Computer-assisted tissue analysis within pathology is not restricted to the quantification and classification of specific tissue entities. DL allows to directly address clinically relevant questions related to the prediction of cancer outcome and efficacy of cancer treatment. This thesis focused on the following crucial research question: is it possible to predict cancer outcome, biomarker status, and treatment efficacy directly from the tissue morphology using DL without any special stains or molecular methods? To address this question, we utilised digitised hematoxylin-eosin-stained (H&E) tissue specimens from two common types of solid tumours – breast and colorectal cancer. Tissue specimens and corresponding clinical data were retrieved from retrospective patient series collected in Finland. First, a DL-based algorithm was developed to extract prognostic information for patients diagnosed with colorectal cancer, using digitised H&E images only. Computational analysis of tumour tissue samples with DL demonstrated a superhuman performance and surpassed a consensus of three expert pathologists in predicting five-year colorectal cancer-specific outcomes. Then, outcome prediction was studied in two independent breast cancer patient series. Particularly, generalisation of the trained algorithms to previously unseen patients from an independent series was examined on the large whole-slide tumour specimens. In breast cancer outcome prediction, we investigated a multitask learning approach by combining outcome and biomarker-supervised learning. Our experiments in breast and colorectal cancer show that tissue morphological features learned by the DL models supervised by patient outcome provided prognostic information independent of established prognostic factors such as histological grade, tumour size and lymph nodes status. Additionally, the accuracy of DL-based predictors was compared to other prognostic characteristics evaluated by pathologists in breast cancer, including mitotic count, nuclear pleomorphism, tubules formation, tumour necrosis and tumour-infiltrating lymphocytes. We further assessed if molecular biomarkers such as hormone receptor status and ERBB2 gene amplification can be predicted from H&E- stained tissue samples obtained at the time of diagnosis from patients with breast cancer and showed that molecular alterations are reflected in the basic tissue morphology and can be captured with DL. Finally, we studied how morphological features of breast cancer can be linked to molecularly targeted treatment response. The results showed that ERBB2-associated morphology extracted with DL correlated with the efficacy of adjuvant anti-ERBB2 treatment and can contribute to treatment-predictive information in breast cancer. Taken together, this thesis shows the potential utility of DL in tissue-based characterisation of cancer for prediction of cancer outcome, tumour molecular status and efficacy of molecularly targeted treatments. DL-based analysis of the basic tissue morphology can provide significant predictive information and be combined with clinicopathological and molecular data to improve the accuracy of cancer diagnostics.Koneoppiminen syväoppimisen (SO) muodossa on muuttanut, miten tietokonenäön tehtävät ratkaistaan monilla toimialueilla, kuten lääketieteellisessä kuvantamisdiagnostiikkassa. SO-perusteiset menetelmät mahdollistavat tarkemman kvantitatiivisen karakterisoinnin syöpäkas- vainnäytteistä, jotka rutiinisti analysoidaan kliinisen patologian laboratorioissa diagnosointia varten. Tietokoneavusteinen kudosanalyysi ei rajoitu ainoastaan tiettyjen kudosentiteettien määrittämiseen ja luokitteluun. SO:n avulla voidaan suoraan tutkia syövän ennustetta ja syöpähoitojen vastetta. Tämä väitöskirja keskittyi tärkeään tutkimuskysymykseen: onko syövän ennuste, biomarkke- rien status ja hoidon tehokkuus mahdollista ennustaa SO:lla suoraan kudosmorfologiasta ilman erillisiä värjäyksiä tai molekyylibiologisia testejä? Vastataksemme tähän kysymykseen käytimme digitaalisia hematoksyliini-eosiini (H&E)-värjättyjä kudosnäytteitä kahdesta taval- lisesta kiinteästä kasvaimesta, rinta- ja paksusuolensyövästä. Kudosnäytteet ja niihin liittyvät kliiniset tiedot saatiin Suomessa kerätystä retrospektiivisestä potilassarjasta. Ensimmäiseksi kehitimme SO-algoritmin, jolla poimimme prognostisen tiedon paksusuolensyöpäpotilaista käyttäen ainoastaan digitalisoituja H&E-värjäyksiä. Kudosnäytteistä SO:lla tehty laskennalli- nen analyysi osoitti ihmisasiantuntijaa parempaa suorituskykyä ja ylitti kolmen patologian asiantuntijan antaman yksimielisen viiden vuoden ennusteen syövän lopputulemasta. Seu- raavaksi lopputuleman ennustamista tutkittiin kahdessa erillisessä rintasyöpäpotilassarjassa. Erityisesti tutkimme koulutetun algoritmin kykyä yleistää syöpäkudosten kokoleikkeistä, jotka olivat peräisin erillisestä algoritmille aiemmin tuntemattomasta potilassarjasta. Rin- tasyövän ennusteen suhteen tutkimme ”multitask learning”-lähestymistapaa yhdistämällä eloonjäämis- ja biomarkkeri-valvotun oppimisen. Tutkimuksemme rinta- ja paksusuolen- syövän osalta osoittavat, että SO-mallien avulla, jotka ovat opetettu potilaan eloonjäämisen mukaan, voidaan kudosmorfologian perusteella saada ennuste, joka on rippumaton aiemmin saatavilla olevista ennustetekijöistä, kuten histologisesta luokittelusta, kasvaimen koosta ja imusolmukkeiden statuksesta. Lisäksi SO-perusteisten ennusteiden tarkkuutta rintasyövässä verrattiin patologien arvioimiin syovän, kuten mitoosien lukumäärä, tuman pleomorfismiin, tubulusten tiehyeiden erilaistumisasteeseen, kasvaimen nekroosiin ja kasvaimen infiltroiviin lymfosyytteihin. Tutkimme myös, voiko rintasyöpäpotilailta syöpädiagnosoinnin yhteydessä saaduista H&E-värjätyistä kudosnäytteistä ennustaa molekulaarisia biomarkkereita, kuten hormonireseptoristatusta ja ERBB2-geenin monistumista. Tutkimuksemme osoitti, että mo- lekulaariset muutokset löytyvät myös kudosmorfologiasta ja ne voi tunnistaa SO:n avulla. Lopuksi tutkimme, miten rintasyövän morfologiset piirteet voidaan yhdistää hoitovasteeseen. Tutkimuksemme osoitti, että SO:n tunnistama ERBB2-positiivisen kasvaimen morfologia kor- reloi anti-ERBB2-liitännäishoitojen tehokkuuden kanssa ja SO:ta voi käyttää ennustamaan rintasyövän lääkevastetta. Tämän väitöskirjatyön tulokset osoittavat, että SO:n syöpäkudoksen karakterisointi voi olla hyödyllinen syövän ennusteen arvioinnissa sekä, molekulaarisen statuksen ja lääkevas- teen ennustamisessa. SO-perusteinen kudosmorfologinen analyysi voi antaa merkittävää tietoa syövän ennusteesta ja se voidaan yhdistää kliiniseen patologiaan ja molekulaariseen informaatioon tarkemman syöpädiagnosoinnin mahdollistamiseksi

Interacting fermions in two dimensions: beyond the perturbation theory

We consider a system of 2D fermions with short-range interaction. A straightforward perturbation theory is shown to be ill-defined even for an infinitesimally weak interaction, as the perturbative series for the self-energy diverges near the mass shell. We show that the divergences result from the interaction of fermions with the zero-sound collective mode. By re-summing the most divergent diagrams, we obtain a closed form of the self-energy near the mass shell. The spectral function exhibits a threshold feature at the onset of the emission of the zero-sound waves. We also show that the interaction with the zero sound does not affect a non-analytic, $T^{2}$-part of the specific heat.Comment: 5 pages, 4 figure

Chiral Spin Waves in Fermi Liquids with Spin-Orbit Coupling

We predict the existence of chiral spin waves collective modes in a two-dimensional Fermi liquid with the Rashba or Dresselhaus spin-orbit coupling. Starting from the phenomenological Landau theory, we show that the long-wavelength dynamics of magnetization is governed by the Klein- Gordon equations. The standing-wave solutions of these equations describe "particles" with effective masses, whose magnitudes and signs depend on the strength of the electron-electron interaction. The spectrum of the spin-chiral modes for arbitrary wavelengths is determined from the Dyson equation for the interaction vertex. We propose to observe spin-chiral modes via microwave absorption of standing waves confined by an in-plane profile of the spin-orbit splitting

Influenza pH1N1 virus accumulated H275Y mutation neuraminidase during propagation in MDCK cells

Peer reviewe

Systems pathology by multiplexed immunohistochemistry and whole-slide digital image analysis

The paradigm of molecular histopathology is shifting from a single-marker immunohistochemistry towards multiplexed detection of markers to better understand the complex pathological processes. However, there are no systems allowing multiplexed IHC (mIHC) with high-resolution whole-slide tissue imaging and analysis, yet providing feasible throughput for routine use. We present an mIHC platform combining fluorescent and chromogenic staining with automated whole-slide imaging and integrated whole-slide image analysis, enabling simultaneous detection of six protein markers and nuclei, and automatic quantification and classification of hundreds of thousands of cells in situ in formalin-fixed paraffin-embedded tissues. In the first proof-of-concept, we detected immune cells at cell-level resolution (n = 128,894 cells) in human prostate cancer, and analysed T cell subpopulations in different tumour compartments (epithelium vs. stroma). In the second proof-of-concept, we demonstrated an automatic classification of epithelial cell populations (n = 83,558) and glands (benign vs. cancer) in prostate cancer with simultaneous analysis of androgen receptor (AR) and alpha-methylacyl-CoA (AMACR) expression at cell-level resolution. We conclude that the open-source combination of 8-plex mIHC detection, whole-slide image acquisition and analysis provides a robust tool allowing quantitative, spatially resolved whole-slide tissue cytometry directly in formalin-fixed human tumour tissues for improved characterization of histology and the tumour microenvironment.Peer reviewe

Deep learning based tissue analysis predicts outcome in colorectal cancer

Image-based machine learning and deep learning in particular has recently shown expert-level accuracy in medical image classification. In this study, we combine convolutional and recurrent architectures to train a deep network to predict colorectal cancer outcome based on images of tumour tissue samples. The novelty of our approach is that we directly predict patient outcome, without any intermediate tissue classification. We evaluate a set of digitized haematoxylin-eosin-stained tumour tissue microarray (TMA) samples from 420 colorectal cancer patients with clinicopathological and outcome data available. The results show that deep learning-based outcome prediction with only small tissue areas as input outperforms (hazard ratio 2.3; CI 95% 1.79-3.03; AUC 0.69) visual histological assessment performed by human experts on both TMA spot (HR 1.67; CI 95% 1.28-2.19; AUC 0.58) and whole-slide level (HR 1.65; CI 95% 1.30-2.15; AUC 0.57) in the stratification into low-and high-risk patients. Our results suggest that state-of-the-art deep learning techniques can extract more prognostic information from the tissue morphology of colorectal cancer than an experienced human observer.Peer reviewe

Breeze : an integrated quality control and data analysis application for high-throughput drug screening

High-throughput screening (HTS) enables systematic testing of thousands of chemical compounds for potential use as investigational and therapeutic agents. HTS experiments are often conducted in multi-well plates that inherently bear technical and experimental sources of error. Thus, HTS data processing requires the use of robust quality control procedures before analysis and interpretation. Here, we have implemented an open-source analysis application, Breeze, an integrated quality control and data analysis application for HTS data. Furthermore, Breeze enables a reliable way to identify individual drug sensitivity and resistance patterns in cell lines or patient-derived samples for functional precision medicine applications. The Breeze application provides a complete solution for data quality assessment, dose-response curve fitting and quantification of the drug responses along with interactive visualization of the results.Peer reviewe

Singular perturbation theory for interacting fermions in two dimensions

We consider a system of interacting fermions in two dimensions beyond the second-order perturbation theory in the interaction. It is shown that the mass-shell singularities in the self-energy, arising already at the second order of the perturbation theory, manifest a non-perturbative effect: an interaction with the zero-sound mode. Resumming the perturbation theory for a weak, short-range interaction and accounting for a finite curvature of the fermion spectrum, we eliminate the singularities and obtain the results for the quasi-particle self-energy and the spectral function to all orders in the interaction with the zero-sound mode. A threshold for emission of zero-sound waves leads a non-monotonic variation of the self-energy with energy (or momentum) near the mass shell. Consequently, the spectral function has a kink-like feature. We also study in detail a non-analytic temperature dependence of the specific heat, $C(T)\propto T^2$. It turns out that although the interaction with the collective mode results in an enhancement of the fermion self-energy, this interaction does not affect the non-analytic term in $C(T)$ due to a subtle cancellation between the contributions from the real and imaginary parts of the self-energy. For a short-range and weak interaction, this implies that the second-order perturbation theory suffices to determine the non-analytic part of $C(T)$. We also obtain a general form of the non-analytic term in $C(T)$, valid for the case of a generic Fermi liquid, \emph{i.e.}, beyond the perturbation theory.Comment: 53 pages, 10 figure

Quantum magneto-oscillations in a two-dimensional Fermi liquid

Quantum magneto-oscillations provide a powerfull tool for quantifying Fermi-liquid parameters of metals. In particular, the quasiparticle effective mass and spin susceptibility are extracted from the experiment using the Lifshitz-Kosevich formula, derived under the assumption that the properties of the system in a non-zero magnetic field are determined uniquely by the zero-field Fermi-liquid state. This assumption is valid in 3D but, generally speaking, erroneous in 2D where the Lifshitz-Kosevich formula may be applied only if the oscillations are strongly damped by thermal smearing and disorder. In this work, the effects of interactions and disorder on the amplitude of magneto-oscillations in 2D are studied. It is found that the effective mass diverges logarithmically with decreasing temperature signaling a deviation from the Fermi-liquid behavior. It is also shown that the quasiparticle lifetime due to inelastic interactions does not enter the oscillation amplitude, although these interactions do renormalize the effective mass. This result provides a generalization of the Fowler-Prange theorem formulated originally for the electron-phonon interaction.Comment: 4 pages, 1 figur

Genome-Wide Analysis of Evolutionary Markers of Human Influenza A(H1N1)pdm09 and A(H3N2) Viruses May Guide Selection of Vaccine Strain Candidates

Here we analyzed whole-genome sequences of 3,969 influenza A(H1N1)pdm09 and 4,774 A(H3N2) strains that circulated during 2009-2015 in the world. The analysis revealed changes at 481 and 533 amino acid sites in proteins of influenza A(H1N1)pdm09 and A(H3N2) strains, respectively. Many of these changes were introduced as a result of random drift. However, there were 61 and 68 changes that were present in relatively large number of A(H1N1)pdm09 and A(H3N2) strains, respectively, that circulated during relatively long time. We named these amino acid substitutions evolutionary markers, as they seemed to contain valuable information regarding the viral evolution. Interestingly, influenza A(H1N1)pdm09 and A(H3N2) viruses acquired non-overlapping sets of evolutionary markers. We next analyzed these characteristic markers in vaccine strains recommended by the World Health Organization for the past five years. Our analysis revealed that vaccine strains carried only few evolutionary markers at antigenic sites of viral hem agglutinin (HA) and neuraminidase (NA). The absence of these markers at antigenic sites could affect the recognition of HA and NA by human antibodies generated in response to vaccinations. This could, in part, explain moderate efficacy of influenza vaccines during 2009-2014. Finally, we identified influenza A(H1N1)pdm09 and A(H3N2) strains, which contain all the evolutionary markers of influenza A strains circulated in 2015, and which could be used as vaccine candidates for the 2015/2016 season. Thus, genome-wide analysis of evolutionary markers of influenza A(H1N1)pdm09 and A(H3N2) viruses may guide selection of vaccine strain candidates.Peer reviewe